1. Statement of the Technical Field
The present invention relates to systems logging and more particularly to the aggregation of log data in a virtual log.
2. Description of the Related Art
Diagnostics forms a critical foundation for the successful operation of a computing system. System elements ranging from individual processes within a computing application to the hardware devices which support the operation of the computing application each can produce volumes of log data. Using the log data, systems analysts, or even automated processes, can analyze the operation of the system for performance monitoring and problem determination and remediation. Accordingly, the production and storage of log data can fulfill an important role in the management of a computing system.
For many decades, individual computing applications and components have been configured to generate log data, particularly in an effort to track the execution progress of the application and the operational performance of the components. The advent and commercial utilization of widespread computing networks, including the global Internet, have encouraged the componentization and distribution of computing applications and systems elements. As a result, tracking the log data for each distributed system element can be burdensome if not impossible.
To facilitate the management of a distributed set of logs produced by a disparate arrangement of systems elements, the virtual log has been developed as a tool for aggregating the log data from a heterogeneous collection of systems elements. As an example, in United States Patent Application Publication US 2003/0226058 published on Dec. 4, 2003 by Thomas Jay Miller et al. for VIRTUAL LOGGING SYSTEM AND METHOD, a common transactional logging system is proposed that presents to one or more log clients the appearance that each log client is interacting with a dedicating logging system. In reality, however, the virtual logging system is multiplexing multiple virtual log streams, including log records, for each log client into a single transactional log. Moreover, the Miller transactional logging system does not account for the different native formats, and particularly the heterogeneous mechanisms used by different logging sources, in performing the logging. The problem of heterogeneous mechanisms will be recognized as a more complex issue than merely that of different logging sources requiring a single abstract representation to consumers.
To track the operation of a computing system through the use of a virtual log, the individual logs of the various and sundry elements of the system can be combined into a single, virtual log. While it appears that combining information from a set of log files in a distributed arrangement into a single virtual log ought to be a straightforward operation, it is not always so as the skilled artisan will recognize. Specifically, log files can be limited in size in a configurable manner. Where a log file is size-limited, the log file may wrap upon itself. This requires an external entity to keep track of which records have already been processed and which records are yet to processed because snapshots of the real log may provide events that have already been processed during the previous processing cycle. Tracking the state of log records in a wrapping log file can be difficult in nature and, when performed improperly, can result in a loss of log data. Of course, where a log file is too large, computing resources will have been wasted.
Given the vast size of the individual logs, a comprehensive real-time aggregation of individual log files into a virtual log can be resource consumptive beyond that which is reasonable. Accordingly, virtual logging systems can periodically sample the logs of each of the individual products. Yet, to establish a sampling interval which is too long, temporally speaking, can result in the repeated wrappings of individual log files causing a loss of data. In contrast, if the sampling interval is set too short, limited computing resources can be unnecessarily wasted in furtherance of a diagnostic goal at the expense of the operation of the computing system.
Whereas optimally configuring a virtual log can be problematic generally, in an autonomic system, the problem can be particularly acute. For the uninitiated, autonomic computing systems self-regulate, self-repair and respond to changing conditions, without requiring any conscious effort on the part of the computing system operator. To that end, the computing system itself can bear the responsibility of coping with its own complexity. The crux of autonomic computing relates to eight principal characteristics:
I. The system must “know itself” and include those system components which also possess a system identify.
II. The system must be able to configure and reconfigure itself under varying and unpredictable conditions.
III. The system must never settle for the status quo and the system must always look for ways to optimize its workings.
IV. The system must be self-healing and capable of recovering from routine and extraordinary events that might cause some of its parts to malfunction.
V. The system must be an expert in self-protection.
VI. The system must know its environment and the context surrounding its activity, and act accordingly.
VII. The system must adhere to open standards.
VIII. The system must anticipate the optimized resources needed while keeping its complexity hidden from the user.
Thus, in keeping with the principles of autonomic computing, it would be preferred if a virtual log could configure and reconfigure itself under varying and unpredictable conditions. Also, it would be preferred if a virtual log never settled for the status quo and always looked for ways to optimize its workings. It would be further preferred if a virtual log could self-heal and recover from routine and extraordinary events that might cause some of its parts to malfunction. It would be yet further preferred if a virtual log could know its environment and the context surrounding its activity, and act accordingly. Finally, it would be even yet further preferred if a virtual log could anticipate the optimized resources needed while keeping its complexity hidden from the user.